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DETAILED ACTION 



Claim Objections 

1 . Claim 17 is objected to because of the following informalities: 

Claim 17 refers back to an unknown claim; claim 38. Appropriate 
correction is required. 



Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 

USPQ 459 (1966), that are applied for establishing a background for determining 

obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

2. Claims 1-5, 7-11, and 14-15 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over Nielsen (U.S. Pat. No. 5,897,644) in view of Betz et al. (Pre Grant 



Pub. 2002/0106191 A1). 
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Regarding claim 1 , Nielsen discloses a method of displaying an interactive 
picture using a markup document (see [column 3 lines 38-40] where a method 
for presenting a (HTML) data stream, also referred to as art image, on a device is 
disclosed [column 5 lines 7-1 1] where HTML can contain graphical data; see 
also [column 10 lines 7-9] where other HTML non-textual elements are 
anticipated and can also be presented). 

Nielsen further discloses obtaining device-aspect-ratio information 
corresponding to the markup document (see [column 7 lines 50-65] for the 
process of accessing information about the parameters of a specific display 
device intended to present the interactive image includes determining the aspect 
ratio). 

Nielsen further discloses obtaining a markup picture from the markup 
document according to the device-aspect-ratio information (see [column 8 line 66 
through column 9 line 5] where the HTML data stream is processed according to 
the predetermined aspect ratio and the resulting HTML image is obtained). 

Nielsen does not explicitly teach displaying audio and/or video (AV) in this 
process comprising obtaining an AV picture from the AV data and displaying the 
interactive picture, which includes combining the AV picture and the markup 
picture. 

Betz however teaches presenting a video or image montage by 
referencing markings which link titles on a digital video or DVD which contain 
individual shots, scenes, or sequences (see [0008-0009] and [0029]). Betz 
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further teaches that the aforementioned media can contain audio as well as video 
(AV) data (see [0034]). 

At the time the invention was made it would have been obvious to a 
person of ordinary skill in the art that the display of a HTML markup language 
containing various HTML textual and non-textual elements, as taught in Nielsen, 
could contain audio and video (AV) elements, as taught in Betz, because AV is a 
form of non-textual element and common place in media and media processing 
systems (see [0004]). 

Regarding claim 2, Nielsen, as applied to Betz, discloses all of the 
limitations of claim 1 including wherein the obtaining of the device-aspect-ratio 
information comprises reading information on an aspect ratio of a target display 
device intended to display the markup document in an interactive mode (see 
[column 7 lines 50-65] for the process of accessing information about the 
parameters of a specific display device intended to present the interactive image 
includes determining the aspect ratio). 

Regarding claim 3, Nielsen, as applied to Betz, discloses all of the 
limitations of claim 1 including wherein the obtaining of the device-aspect-ratio 
information comprises reading information on an aspect ratio of a target display 
device intended to display the markup document in an interactive mode 
according to a design of a markup document maker (see [column 7 lines 50-65]; 
see also [column 3 lines 29-37] the HTML image is presented in the original 
proportions as the author intended). 
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Regarding claim 4, Nielsen, as applied to Betz, discloses all of the 
limitations of claim 1 including wherein the obtaining of the device-aspect-ratio 
information comprises parsing the device-aspect-ratio information which is 
written in the markup document using a property of a tag (see [column 7 lines 1 - 
12] where the obtaining the canvas specification comprises a parsing mechanism 
and using unique HTML markup tags). 

Regarding claim 5, Nielsen, as applied to Betz, discloses all of the 
limitation of claim 1 including wherein the obtainment of the markup picture 
comprises transforming a source markup picture into the markup picture with a 
predetermined aspect ratio according to the device-aspect-ratio information, the 
source markup picture being obtained from the markup document (see [column 7 
line 66-column 8 line 4] where the HTML data stream is transformed, according 
to a predetermined aspect ratio, and presented as resultant HTML image; see 
also [column 5 lines 7-1 1] where the HTML data stream can obtain graphics 
from graphical data; see also [column 10 lines 7-9] where other HTML non- 
textual elements can be transformed). 

Regarding claim 7, Nielsen, as applied to Betz, discloses all of the 
limitations of claim 1 including wherein the obtainment of the markup picture 
comprises scaling a source markup picture to output the markup picture 
corresponding to the device-aspect-ratio information in response to information 
on an aspect ratio of a destination device being different form the device-aspect- 
ration information, the destination device substantially displaying the markup 
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document and the source markup picture being obtained from the markup 
document (see [column 7 line 66 through column 8 line 5] where the HTML 
image can be reduced or magnified according to the transformation limitations; 
see also [column 8 lines 20-26} for the maximization in the device accessible 
area). 

Regarding claim 8, Nielsen, as applied to Betz, discloses all of the 
limitations of claim 7, but does not explicitly teach wherein the destination device 
is a television monitor. 

Betz teaches using a television as a display (see [0034]). 

At the time the invention was made, it would have been obvious to a 
person of ordinary skill in the art to display an HTML data stream image, as 
taught in Nielsen, on a television, as taught in Betz because televisions are 
commonly used to view media presentations (see [0004]). 

Regarding claim 9, Nielsen, as applied to Betz, discloses all of the 
limitations of claim 1 including wherein the device-aspect-ration information is 
included in the markup document (see [column 3 lines 40-44] where the device- 
aspect-ratio information, referred to herein as the canvas specification, is 
obtained from within the HTML data stream). 

Regarding claim 10, Nielsen, as applied to Betz, discloses all of the 
limitations of claim 1 including wherein the displaying of the interactive picture 
further includes obtaining embedding information included in the markup 
document to combine the AV picture with respect to the markup picture (see 
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[column 3 lines 40^4] where the information is obtained from within the HTML 
data stream; see also [column 5 lines 7-11] where the HTML data stream can 
contain graphics obtained through processing graphical data; see also [column 
10 lines 7-9] where other HTML non-textual elements, such as AV pictures in 
light of the motivation for claim 1 , are transformed and displayed). 

Regarding claim 1 1 , Nielsen, as applied to Betz, discloses all of the 
limitations of claim 1 including wherein the obtaining of the device-aspect-ratio 
information comprises reading information on an aspect ratio of a target display 
device intended to display the markup document and information on a resolution 
of the markup document, the information on the aspect ratio and the information 
on the resolution being written in the markup document using a property of a tag 
(see claim 2 rejection; see also [column 9 line 58 through column 10 line 6] 
where the resolution, number of pixels in the view area, is stored in the markup 
document under the property tag "AREA"). 

Regarding claim 14, Nielsen discloses a method of processing a source 
markup picture of a markup document and a source audio video (AV) picture of 
an AV data in an interactive mode (see [figure 3] for a method of processing an 
HTML data stream to output a HTML image; see also [column 6 line 26-40] to 
one of ordinary skill in the art it is obvious that a general computer is capable of 
processing AV data). 

Nielsen further discloses obtaining AV picture typed selection information 
(see [column 9 lines 45-57] where the graphic or referenced media type is 
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selected based on the HTML element tag; see also [column 6 line 26-40] it is well 
known in the art that the general computer described is capable of obtaining AV 
data from an HTML reference element in the same manner as obtaining the 
graphic element); determining whether device-aspect-ratio information is 
included in the markup document (see [column 3 lines 38-44] for detecting 
aspect ratio of markup document) ; transforming the source AV picture and the 
source markup picture according to the AV picture type selection information and 
the device-aspect-ratio information, respectively, in response to the device- 
aspect-ratio information being included in the markup document (column 3 lines 
38-50] for transformation; see also [column 7 lines 50-53] for obtaining the 
device-aspect-ratio information). 

Nielsen does not teach transforming the source AV picture according to 
the AV picture type selection information and outputting the source markup 
picture as is in response to the device-aspect-ratio information not being included 
in the markup document. 

However, at the time of the invention it would be obvious, to one of 
ordinary skill in the art, to display the HTML image "as is" when the device- 
aspect-ratio information is included or not as opposed to transforming its 
dimensions, as taught in Nielsen (see [figure 3]) because displaying the HTML 
image "as is" due to the device-aspect-ratio being available or not can be 
accomplished by simply not applying one or more pre-existing processes. 



Application/Control Number: 10/685,701 Page 9 

Art Unit: 2609 

Regarding claim 15, Nielsen discloses a computer readable medium 
encoded with operation instructions for implementing a method of displaying an 
interactive picture using a markup document, performed by a computer (see 
[column 4 lines 20-29]). 

Nielsen further discloses obtaining device-aspect-ratio information 
corresponding to the markup document (see [column 3 lines 38-44] contents of 
markup document obtained and used). 

Nielsen further discloses obtaining a markup picture from the markup 
document according to the device-aspect-ratio information (see [column 7 line 
66-column 8 line 4] where the HTML data stream is transformed, according to a 
predetermined aspect ratio, using a presentation mechanism to present a 
resultant HTML image; see also [column 5 lines 7-1 1] where the HTML data 
stream can obtain graphics from graphical data; see also [column 10 lines 7-9] 
where likewise other HTML non-textual elements can be obtained and 
transformed). 

However, Nielsen does not explicitly teach a computer readable medium 
encoded with operation instructions for implementing a method of displaying an 
interactive picture using audio video (AV) data, performed by a computer 
comprising obtaining an AV picture from the AV data and displaying the 
interactive picture including combining the AV picture and the markup picture. 

Betz teaches presenting a video or image montage by referencing 
markings which link titles on a digital video or DVD which contain individual 
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shots, scenes, or sequences (see [0008-0009] and [0029]). Betz further teaches 
that the aforementioned media can contain audio as well as video (AV) data (see 
[0034]). 

At the time the invention was made it would have been obvious to a 
person of ordinary skill in the art to obtain and combine AV elements, as taught in 
Betz, for the use and integration with the HTML markup processing, as taught in 
Nielsen, because the AV elements like the graphical elements in Nielsen are 
common media elements and often used in HTML markups (see Betz [0008]). 
3. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over Nielsen 
(U.S. Pat. No. 5,897,644)h :^p*4 -Betz et al. (Pre Grant Pub. 2002/0106191 A1), 
in view of Yamauchi etal. (U.S. Pat. No. 5,907,659). 

Nielsen, as applied to Betz, discloses all of the limitations of claim 1, 
including wherein the obtainment of the markup picture comprises transforming a 
source markup picture into some markup picture according to the device-aspect- 
ratio information, the source markup picture being obtained from the markup 
document (see claim 5 rejection). 

Nielsen does not explicitly teach transforming said source markup picture 
into a markup picture with an aspect ratio of 4:3 or 1 6:9. 

Yamauchi teaches the transformation to aspect ratios of 4:3 and 1 6:9 (see 
[column 2 lines 19-49]). 

At the time the invention was made it would have been obvious, to a 
person of ordinary skill in the art, to transform an HTML data stream image, as 
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taught in Nielsen, into an image with an aspect ratio of 4:3 or 16:9 because these 
aspect ratio standards are widely accepted and used. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

4. Claim 12 is rejected under 35 U.S.C. 102(b) as being anticipated by Nielsen 
(U.S. Pat. No. 5,897,644). 

Regarding claim 12, Nielsen discloses a method of processing a markup 

document in an interactive mode (see [column 3 line 38-50] for the general 

method). 

Nielsen further discloses determining whether device-aspect-ratio 
information is included in the markup document (see [column 3 lines 38-44] for 
detecting aspect ratio of markup document). 

Nielsen further discloses the device-aspect-ratio information being data 
including information on an aspect ratio of a target display device intended to 
display the markup document in the interactive mode (see [column 3 lines 44- 
45]; see also [column 7 lines 50-53] for disclosure regarding the target display 
device). 
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Nielsen further discloses processing the markup document according to 
the device-aspect-ratio information in response to the device-aspect-ratio 
information being included in the markup document (see [column 7 lines 50-57] 
where the markup application is processed according to the limit allowed by the 
determined aspect ratio values). 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over Nielsen 

(U.S. Pat. No. 5,897,644). 

Regarding claim 13, Nielsen discloses all of the limitations of claim 
12 including wherein the processing for the markup document includes 
transforming a source markup picture obtained from the markup document 
according to the device-aspect-ration information in response to information on 
an aspect ratio of a destination device displaying the markup document being 
different from the device-aspect-ratio information (see [column 7 line 66 through 
column 8 line 5] where the HTML image can be reduced or magnified according 
to the transformation limitations obtained prior to displaying; see also [column 8 
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lines 20-26] see also [column 5 lines 7-il] where the HTML data stream can 
obtain graphics from graphical data). 

Nielsen further discloses outputting the source markup picture once 
transformed in response to the information on the aspect ratio of the destination 
device corresponding to the device-aspect-ratio information (see [317 and 319 of 

v 

figure 3]). 

At the time of the invention it would be obvious, to one of ordinary skill in 
the art, to display the HTML image "as is" as opposed to transforming its 
dimensions because displaying the HTML image "as is" can be accomplished by 
simply not applying one or more pre-existing processes. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

6. Claims 16 and 18 are rejected under 35 U.S.C. 102(b) as being anticipated by 

Nielsen (U.S. Pat. No. 5,897,644). 

Regarding claim 16, Nielsen discloses a computer readable medium 
encoded with operating instructions for implementing a method of processing a 
markup document in an interactive mode, performed by a computer (see [column 
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4 lines 20-29] where a program is embedded on a computer usable medium for 
the expresses purpose of processing a markup HTML data stream). 

Nielsen further discloses determining whether device-aspect-ratio 
information is included in the markup document, the device-aspect-ratio 
information being data including information on an aspect ratio of a target display 
device intended to display the markup document in the interactive mode (see 
[column 4 lines 20-29]; see also [column 7 lines 50-65] where the process of 
accessing information about a specific display device, intended to present the 
interactive capable markup data, includes determining the aspect ratio). 

Nielsen further discloses processing the markup document according to 
the device-aspect-ration information in response to the device-aspect-ration 
information being included in the markup document (see [column 4 lines 20-29]; 
see also [column 7 lines 50-57] where the markup application is processed 
according to the limit allowed by the determined aspect ratio values). 

Regarding claim 18, Nielsen discloses a method in a computer system to 
. process audio video (AV) data in an interactive mode using a markup document, 
the method comprising: controlling an AV playback engine to obtain a video 
picture of the AV data; and controlling an enhanced audio and video (ENAV) 
engine to transform a markup picture of the markup document according to 
device-aspect-ratio information corresponding to the aspect ratio of the target 
display device intended to display the markup document in the interactive mode 
(see [column 3 lines 38-50] for a computer controlled method; see also [column 7 
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line 66 through column 8 line 5] where method is capable of transforming HTML 
data into an HTML image; see al,so [column 7 lines 50-53] where method is able 
to obtain the aspect ratio of the target display; see also [column 6 line 26-40] for 
a description of a general computer which is capable of processing AV as well as 
ENAVdata). 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over Nielsen 

(U.S. Pat. No. 5,897,644). 

Regarding claim 17, with the assumption this claim was intended to refer 
back to claim 16, Nielsen discloses all of the limitations of claim 16 including 
wherein the processing of the markup document includes transforming a source 
markup picture obtained from the markup document according to the device- 
aspect-ratio information in response to information on an aspect ratio of a 
destination device displaying the markup document being different from the 
device-aspect-ratio information 

Nielsen further discloses outputting the source markup picture once 
transformed in response to the information on the aspect ratio of the destination 
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device corresponding to the device-aspect-ratio information (see [317 and 319 of 
figure 3]). 

At the time of the invention it would be obvious, to one of ordinary skill in 
the art, to display the HTML image "as is" as opposed to transforming its 
dimensions because displaying the HTML image "as is" can be accomplished by 
simply not applying one or more pre-existing processes. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jason Thomas whose telephone number is (571) 270- 
5080. The examiner can normally be reached on Mon. - Thurs., 8:00a.m. - 5:00p.m. 
EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Dennis Chow can be reached on (571) 272-7767. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Jason Thomas 



DENNIS DOON CHOW 
SUPERVISORY PATENT EXAMINER 




